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SUMMARY 


Two versions of a real-time digital- computer program that operates a 
fighter airplane interactively against a human pilot in simulated air combat 
have been evaluated. They function by replacing one of two pilots in the 
Langley differential maneuvering simulator. Both versions make maneuvering 
decisions from identical information and logic; they differ essentially in the 
aerodynamic models that they control. One is very complete, but the other is 
much simpler, characterizing primarily the airplane's performance (lift, drag, 
and thrust). Both models competed extremely well against highly trained U.S. 
fighter pilots. Although data from the evaluation indicate that the more com- 
plete model outperformed the pilots to a greater degree than did the simpler 
one, evidence is not sufficient to conclude that the complete model is inher- 
ently superior. 


INTRODUCTION 

In support of the studies in advanced aircraft maneuvering concepts being 
conducted in the Langley differential maneuvering simulator (DMS) , Langley 
Research Center has pursued the development of a logic to effectively maneuver 
an airplane in simulated air combat against a human pilot. This effort has 
resulted in a real-time digital- computer program known as the adaptive maneuver- 
ing logic (AML) program. The AML can be used to replace one of the DMS pilots 
and to provide the remaining pilot with a tough, competitive, realistic adver- 
sary. The AML is completely deterministic, yet because its maneuvering is a 
complex interaction between its own present state and that of its opponent's 
immediate past, it minimizes, if not completely eliminates, any appearance of 
predictability. In fact, the likelihood of any two AML runs against a human 
pilot being identical is no greater than that of the human pilot identically 
repeating a sequence of maneuvers. 

George H. Burgin of Decision Science, Inc., under contract to Langley, 
first formulated the most basic concepts of the AML and demonstrated their func- 
tioning in a batch- processing digital-computer program. It was originally 
intended that those batch programs be used to provide preliminary indications 
of the results of studies to be conducted with human subjects in the DMS. How- 
ever, as the AML development progressed, it became increasingly evident that 
it could be adapted for real-time operation against human pilots. The airplane 
models originally driven by the AML primarily characterized airplane performance 
(lift, drag, and thrust), with minimal modeling of the airplane's rotational 
characteristics. These simplified airplane models were generally considered 
to be acceptable, particularly in light of the more important need at that time 
to enhance the maneuver logic and achieve real-time operation. When real-time 
operation was ultimately achieved, the AML was enthusiastically accepted by 
the fighter pilots who flew against it. However, because the airplane model 
controlled by the pilots was different from that controlled by the AML, a ques- 
tion persisted as to whether the AML could perform maneuvers that the pilots 
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could not. Thus, a new effort was begun to develop a control system which would 
allow the AML to control the same airplane model controlled by the human pilots 
in the DMS. The magnitude of this new program would have far exceeded the capac- 
ity of the already overburdened Control Data series 6600 computer which was 
operating the combined AML-DMS program. Fortunately, the more powerful Control 
Data CYBER series 1 75 computer became available at about the same time the new 
program was ready and was able to easily handle it. 

With the new "control model" available, it was desirable to compare it with 
the older "performance model" since a favorable comparison would increase confi- 
dence in the validity of results obtained in previous studies which had used 
the performance model. It would also help justify the continued use of perfor- 
mance models for most applications. Their continued use is desirable because 
they are much simpler and more readily programmed for additional airplanes. 
Presently, only the F-4 airplane has been fully modeled with a control system 
for operation with the AML. 

During late January 1978, four current U.S. Air Force and U.S. Navy fighter 
pilots flew against an improved maneuver logic that controlled both a perfor- 
mance model F-4 and a control model F-4. It is the primary purpose of this 
paper to report the results of that study. The data analysis will attempt to 
assess the maneuvering effectiveness of both the AML models relative to current 
U.S. military fighter pilots and indirectly to each other. 

Detailed descriptions of the AML and associated airplane models may be 
found in references 1, 2, 3, and 4. This paper will not attempt to repeat the 
information contained in these publications, but will merely summarize in the 
body of the paper some of the more basic concepts, it will use two appendixes 
to provide more in-depth explanations of previously unpublished modifications 
and improvements made to the AML. Appendix A, entitled "Performance-Model 
Modifications," was authored by Bobby J. Glover of Kentron International, Inc., 
whose efforts have constituted an invaluable input to Langley's AML development 
work. Appendix B describes real-time implementation and refinement of the AML 
control model. 


SYMBOLS 

a n,c commanded normal acceleration, g units (1 g = 9.8 m/sec 2 ) 

b wing span, m 

C D drag coefficient 

Cl lift coefficient 

ac L 

C Lr , - , per degree 

a 3a 

Cj rolling-moment coefficient 


2 
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per degree 


C *6 S p 

Cm 

Si 


8C 2 

3B 


rolling moment due to aileron deflection 
rolling moment due to spoiler deflection 
pitching- moment coefficient 
3 C m 

= , per degree 

3q 


g 

h 

x xx 

M 


q 

s 

t 


u,v,w 


w 

x i>Yi ' z i 


X,Y,Z 


a 


B 

6 a^e^r 

^sp 

X 


<t> 


thrust command, N 

acceleration due to gravity, nv/sec 2 
altitude, m 

moment of inertia about X body axis, kg-m 2 
Mach number 

roll, pitch, and yaw angular velocities, respectively, rad/sec 
dynamic pressure, N/m 2 
wing area, m 2 
time, sec 

components of velocity along X, Y, and Z body axes, respectively 
ny'sec 

weight 

coordinates of inertial axis system 
body axes 

angle of attack, deg 
angle of sideslip, deg 

aileron, elevator, and rudder deflections, respectively, deg 
spoiler deflection, deg 
of f-boresight angle, deg 
bank angle, deg 



<t> c bank-angle command, deg 

4 », 0 , 4 > Euler yaw, pitch, and roll angles, respectively, of body axis system 
referenced to inertial axis system 

Euler yaw, pitch, and roll angles, respectively, of maneuver plane 
axis system referenced to inertial axis system 

Abbreviations : 

ACM air- combat maneuvering 

AML adaptive maneuvering logic 

AMLS adaptive-maneuvering-logic score 

DMS Langley differential maneuvering simulator 

MMP moving-maneuver-plane axis system 

TOA time on offense with advantage 

A dot over a symbol indicates derivative with respect to time. 


LANGLEY DIFFERENTIAL MANEUVERING SIMULATOR 

The AML program operates in real time on the DMS which has been in opera- 
tion for about 8 years. It permits two pilots to maneuver their simulated air- 
planes interactively and has been used extensively in performance evaluations 
of fighter-type airplanes in a one-on-one environment. 

Figure 1 is a diagram of the DMS. It consists of two 40-foot-diameter pro- 
jection spheres, with cockpits located in such a way as to position the pilot's 
eye at the center of the sphere. The projection equipment, which is located 
above the rear of the cockpits, projects images of the sky. Earth, horizon, and 
opposing aircraft onto the spheres. The resulting projected images move in 
proper perspective for each of the two pilots. The state of the pilot's air- 
craft is revealed through cockpit instrumentation, visual displays, and secon- 
dary cues such as g-suit, blackout, altitude warning, buffet, and thrust 
noise. 

Figure 2 is a block diagram illustrating the operation of the DMS with two 
pilots. The human pilot reacts to information displayed in his sphere and out- 
puts stick, rudder-pedal, and throttle commands. These are transmitted to the 
equations of motion which compute the state of the aircraft corresponding to 
them. The equations of motion consist of both force and moment equations that 
use tabulated aerodynamic derivatives which define the aircraft being flown. 

The two aircraft may be the same or dissimilar. The relative states of the 
aircraft are computed, and that information is used to drive the display sys- 
tems in the spheres. Additional information may be obtained from reference 5. 
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AML PROGRAMS 


The AML programs are digital- computer programs which interface a guidance 
law for interactive air-combat maneuvering with computer models of fighter air- 
planes. These programs may act as real-time opponents for human pilots in the 
DMS or they may control two math-modeled airplanes against each other in an off- 
line batch- processing mode. The programs are deterministic, giving their users 
ready access to the causes of all actions taken by the AML. 

The logic for selecting maneuvers is common to all the AML programs. How- 
ever, two different types of airplane math models are used with this logic. 

They are called the performance model and the control model. The differences 
between these two forms of the AML program will be illustrated by comparing and 
contrasting how each of them operates when used in conjunction with the DMS. 

Figure 3 illustrates how the DMS works when a pilot flies against the AML 
performance model. The human-pilot loop is exactly as it was in figure 2. The 
block labeled air-combat maneuvering (ACM) logic is the heart of the AML pro- 
gram. This logic will be described later. Note that two basic simplifications 
have been made with respect to the human-pilot loop. First, the ACM logic out- 
puts bank-angle and load-factor commands directly to the equations of motion 
rather than stick and rudder-pedal displacements. Second, rather than using 
moment equations, the canputer-controlled airplane is driven directly in atti- 
tude through its body rates. Filters and rate limits are used to insure smooth 
transitions. Because the airplane is modeled primarily through its performance 
characteristics (lift, drag, and thrust), this form of the AML program is called 
the performance model. 

A second form of the AML program, called the control model, contains a more 
ccmprehensive airplane model (fig. 4) . Note now that the equations of motion 
in the AML loop are identical to those in the human-pilot loop. What is differ- 
ent is the addition of a control system to convert the AML bank-angle and load- 
factor commands to control surface commands. Because of the time required and 
the difficulty of designing this control system, the control-model form of the 
AML has to date been assembled only for the F-4 airplane. This model enjoys 
the advantage of greater user confidence in the realism and accuracy of the 
simulation. It also, in contrast to the performance model, eliminates the 
question of whether the canputer-controlled airplane can perform maneuvers that 
the pilot, at least in theory, cannot perform. 

Rather than use classical maneuvers such as a high-speed yo-yo, the AML 
uses simpler, more elemental maneuvers. These elemental maneuvers consist of 
segments of circular flight paths lying in planes. The plane curve is specified 
by the throttle setting and the percentage of available load factor. The bank 
angle is chosen such that the curved path remains in the desired plane. The 
airplane continues to fly in the specified plane until a new maneuver is chosen. 
When this occurs, the airplane transitions to a new plane of flight by rolling 
until its wings are properly alined to cause all net forces to lie in the new 
plane. The flight then continues. Figure 5 illustrates how several elemental 
segments may combine to form a three-dimensional space curve. 


5 



To maintain flexibility in reevaluating the tactical situation for possible 
manuever changes, the time interval between maneuver decisions must be short. 

The AML uses 1-sec intervals. When the elemental maneuvers are put together, 
they quite often resemble classical maneuvers. 


Maneuver Selection Logic 

Figure 6 is a simplified flow chart of the AML program. The maneuver- 
selection logic is the section of the chart contained between the dashed lines. 

The program uses a fixed logic to decide which maneuver to make next. The 
concept is very simple. The program predicts the future state of its opponent 
over a short time interval. It then computes what its state would be if it per- 
formed each of several optional maneuvers over the same time period. The com- 
puted state of each trial maneuver is compared with the predicted state of the 
opponent, and the most promising trial maneuver is chosen as the next to be 
performed. 

When the program makes maneuvering decisions, it evaluates several stan- 
dard maneuvers, depending upon its assessment of its current tactical situation. 
The maneuvers being discussed here are those considered applicable to a "normal 
situation." The methods employed to handle special problems such as ground 
avoidance and energy management will be discussed later. Under normal circum- 
stances the program evaluates some or all of the following maneuvers: (1) con- 

tinuing the maneuver currently being performed, (2) a maximum load-factor turn 
in the current plane, (3) maximum load-factor turns in planes banked 10° to 
either side of the current plane of flight, (4) flying straight, (5) maximum 
load-factor turns in the "intercept maneuver plane" (the intercept maneuver 
plane is defined as the plane which contains the current velocity vector of 
the computer-controlled airplane and the predicted position of the opposing air- 
plane), (6) maximum load-factor turns in planes banked 10° to either side of 
the intercept maneuver plane, (7) maximum load-factor turns in planes banked 
90°, 180°, and 270° from the intercept maneuver plane, and (8) a turn with a 
load factor computed to yield a circular flight path in the intercept maneuver 
plane which will exactly intercept the opponent at his predicted position. The 
AML extrapolates the opponent's position (and other state variables) at the end 
of a preselected time interval, nominally 2 sec, from the current time using a 
second-order curve containing the opponent's present and two previous positions. 

For each trial maneuver, the program computes what the state of the air- 
plane under its control would be at the end of 2 sec. The computed state vari- 
ables resulting from each trial maneuver are then compared with the predicted 
state variables of the opponent by means of a set of questions which results 
in a numerical score, and the maneuver with the highest score is chosen as the 
next to be performed. 


Scoring Trial Maneuvers 

The following 14 questions are those used to obtain the numerical score 
for each trial maneuver. The questions are worded in such a way that they can 
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be answered either "yes" or "no." A yes answer is considered good and is 
assigned a value of 1 . A no answer is assigned a value of 0. 


Answers 
Yes No 


1. Is opponent ahead of attacker? 1 

2. Is attacker behind opponent? 1 

3. Can attacker see opponent? 1 

4. Is opponent uable to see attacker? 1 

5. Is attacker in certain cone behind opponent? 1 

6. Is opponent not within certain cone behind attacker? 1 

7. Can attacker fire at opponent? 1 

8. Can opponent not fire at attacker? 1 

9. Are both answers to questions 1 and 2 yes? 1 

10. Is attacker closing on opponent? 1 

11. Is attacker's predicted altitude greater than 91.44 m? a 0 

12. Are both the answers to questions 3 and 4 yes? 1 

13. Is attacker's line of sight less than 60°? 1 

14. Is attacker's line of sight zero or decreasing? 1 

SCORE 13 


0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

a -l 3 
0 
0 

0 

-13 


a A "yes" receives a zero; a "no" receives -13. 


Although stated here in words, the questions actually evaluate specific 
relative-state characteristics. That is, the questions involve such quantities 
as positions, angles, velocities, and distances. The reader should note in 
studying these questions that the airplane controlled by the computer model is 
referred to as the attacker. A detailed explanation of each question will not 
be given here. Two questions will be discussed, however, as typical of the 
information being evaluated. Question 1 asks whether the opponent is in front 
of the attacker. Stated in geometric terms, the question asks whether the angle 
between the attacker's X body axis and the line of sight to the opponent's 
center of gravity is less them 90°. A slightly more complex example is ques- 
tion 5, which emks whether the attacker is within a certain cone behind the 


7 


opponent. Specifically, the program determines whether the attacker is in a 60° 
cone behind the opponent at a range of less than 914.4 m. All 14 questions are 
evaluated for each trial maneuver to obtain the numerical score. 

The idea of flying in maneuver planes is more a goal with the AML them it 
is a fact. It provides an especially good mechanism through which to set up 
trial maneuvers and to evaluate them. Because of the mathematical scheme used 
to simulate airplane motion in the performance model, the performance model 
probably comes closer to achieving piecewise planar flight than does the control 
model. The transitioning between maneuver planes may require a large part of 
the time between decisions. During this time, the airplane's flight path is 
obviously not planar. In addition, it is often not possible to maintain planar 
flight once it has begun. The control model uses the maneuver- plane concept 
only at maneuver- dec is ion intervals, and then only to specify a bank angle and 
a percentage of available load factor to use until the decision time occurs 
again. 


Ground Avoidance and Energy Management 

In addition to the basic concepts and guidance principles already dis- 
cussed, there are two other major problems that the maneuvering logic must 
solve. One of these, as previously mentioned, is ground avoidance. The program 
continuously compares the flight-path angle of the computer-controlled airplane 
with tabulated values of the maximum dive angles from which the airplane can 
recover without crashing. As the flight- path angle approaches an unrecoverable 
value, the trial maneuvers that are permitted become more and more limited until 
only a maximum load-factor pull-up in the vertical plane remains. Considerable 
care must be taken in structuring the logic to prevent crashes because ground- 
avoidance maneuvers generally are not the most effective to use against the 
opposing airplane. 

Energy management is a second area of concern. The particular reference 
here is not to optimizing energy management but, more specifically, to prevent- 
ing an airplane from flying so slowly that it can no longer maneuver effec- 
tively. The program continuously monitors the load factor available to the 
computer-controlled airplane. When it diminishes to certain predetermined 
levels, the normal trial maneuvers are modified or different ones are brought 
into consideration. At some levels of available load factor, the airplane is 
restricted to maneuvers which do not exceed certain percentages of the airplane 
sustained load-factor capability. Under the worst of circumstances, the air- 
plane may be required to relax load factor entirely. As with the ground- 
avoidance maneuvers, considerable care must be taken in devising the criteria 
for the use of energy-management maneuvers. Indiscriminate use of either type 
of these special maneuvers could seriously reduce interaction with the opponent. 
The logic gives priority to ground avoidance, energy management, and normal 
maneuvers, in that order. When special maneuvers are used, generally more than 
one is evaluated by using the same selection process as is used with the normal 
maneuvers. Thus, the program at least tends to choose maneuvers which keep the 
airplane turning in the direction of the opponent. 
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Although the program specifies a throttle setting to be used for each trial 
maneuver, whether it be standard, ground avoidance, or energy management, situ- 
ations occur during the execution of maneuvers in which the throttle setting 
should be changed. Three basic situations are covered. First, a reduction in 
throttle setting may be required to prevent the computer- con trolled airplane 
from overshooting its opponent. Speed brakes may also be employed under these 
circumstances. Second, a similar throttle change may be used to induce an over- 
shoot by the opponent. Finally, the throttle is manipulated to cause the air- 
plane to fly at or near its corner velocity (velocity at which the maximum lift 
capability of an airplane equals its structural load limit) . 


Lag and Lead 

To simplify the earlier explanation of the AML evaluation of the predicted, 
relative future states of its own airplane and that of its opponent, a nominal 
value of 2 sec was specified for the prediction and extrapolation intervals. 
These values are seldom actually used, however. An important part of ACM 
strategy involves the employment of lags and leads. If the future state of the 
opponent is predicted over the same time interval that the future state of the 
AML- con troll ed airplane is extrapolated, the situation is near pure pursuit. 
Actually, it is a slight lead since collision- course trajectories are being 
compared. 

If the AML extrapolates its airplane state over a longer time interval than 
it predicts its opponent airplane state, effectively it evaluates lag. Con- 
versely, if the relative lengths of these time intervals are reversed, lead 
maneuvers are evaluated. 

For most situations, the AML uses lag rather than higher risk lead maneu- 
vering. Unless irrevocably committed to a given maneuver, an opponent who 
recognizes that his adversary is using lead pursuit may be able to readily alter 
his trajectory to put the adversary in a disasterous situation. Only in the 
case of equal (2-sec) extrapolation and prediction intervals does the AML even 
approach the application of lead. The degree of lag used varies with the degree 
of positional advantage enjoyed by the AML and the range between the airplanes. 
Large lags, for instance, are employed at very close ranges to prevent 
overshoots. 

Finally, the AML uses long-range as well as short-range prediction and 
extrapolation. The longer the time interval over which the opponent's state 
is predicted, the less accurate it is likely to be because the opponent has 
more time to alter his trajectory. However, at long ranges where errors are 
less critical and softer maneuvering is more likely, longer prediction intervals 
may be safely used to devise longer term, less energy consumptive maneuvers. 

Even with the longer time-interval predictions, the AML still uses lag- type 
maneuvers. Long-term prediction (6 sec) is always used for ranges greater than 
1524 m and is also used for ranges greater than 762 m if the AML has already 
acquired a large angular advantage. 
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Ope rational Modes 

Two basic operational modes of the AML program exist, the off-line or 
batch-processing mode and the real-time simulation mode. The batch-processing 
mode controls two airplanes and supplies maneuvering logic, relative geometry 
computations, equations of motion, and aerodynamic characteristics of the two 
airplanes. Since the same maneuvering logic is used for both airplanes, mea- 
sures of the relative maneuvering capability of two dissimilar airplanes can 
be obtained. The program can also be used to obtain preliminary indications 
of the outcome of pilot- versus-pilot studies. 

The real-time simulation mode utilizes the DMS to allow the computer- 
controlled airplane to fly interactively one-on-one air-combat engagements 
against human pilots. Since the program is deterministic, it can act as an 
invariant or standard opponent whose logic and maneuvering performance can be 
kept constant from one simulator run to another, thus allowing variations in 
human-pilot performance to be measured with respect to it. The AML, in con- 
junction with a manned simulator, can offer significant potential in pilot 
training and proficiency maintenance. 

A more comprehensive explanation of the AML program, including some 
detailed explanations of specific subroutines of the computer program, can be 
found in references 1 and 2. 


DESCRIPTION OF TESTS 
Objectives 

A general procedure was developed to acquire data for evaluation of the 
two AML models. Three basic sets of data would be taken. The first set would 
be recorded from runs in which pilots flew only against other pilots in the 
DMS, and would provide a baseline with which to compare the remaining data. 

The other two data sets would consist of runs in which the same pilots flew 
against each of the two AML models. From analyses and evaluation of these data, 
an assessment would be made of the skill and effectiveness of the AML models 
relative to those fighter pilots. In addition, the maneuvering effectiveness 
of the two models relative to each other would be evaluated. 


Procedure 

A simulated F-4 was flown by both the pilots and the AML. The F-4 was 
chosen because it had been used as the baseline airplane throughout most of the 
development work on the AML and hence was the one most effectively driven by 
the AML. Also, it was the only airplane for which a control system had been 
designed to permit control through its full equations of motion. Furthermore, 
to eliminate the characteristics of the airplane itself as a variable in data 
comparisons, it was necessary to make all runs with the same airplane. 

All runs were made in the DMS. Twelve sessions (1 session per day) of 
3 hr each were conducted over a period of slightly more than 2 weeks. Pilot- 
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versus-pilot runs utilized both DMS spheres while pi lot- vers us- AML runs utilized 
only one. Each data set consisted of 24 runs. 

Run schedules were structured to minimize the effects of extraneous vari- 
ables such as the order in which a given pilot made his runs. Each pilot flew 
four runs against every other pilot during the pilot- versus-pilot data set. 

These runs were ordered such that each pilot flew an equal number of runs in 
each of the two DMS spheres. In each of the AML-versus-pilot data sets, each 
pilot flew against the AML six times. Once a data set began, a log was main- 
tained to show the length of each run, the identification of the pilots, the 
cause of run termination, and other such pertinent information. 

From previous experience with the DMS, it was expected that after about 
three practice sessions, the pilots would be comfortable in the simulator, and 
learning processes with respect to it would have reached a plateau. This was 
expected to be especially true by the time data sets against the AML were begun. 
Thus, no practice runs against the AML were scheduled, and data runs were begun 
immediately upon introducing pilots to the AML models. 

The same initial conditions were used for all runs. The airplanes were 
started head-on from a range of 3.6565 km. The initial altitude of both was 
4.572 km and both had initial speeds of M = 0.9. Each data run was scheduled 
to last 3 min. 

Sixty-eight variables were recorded every 0.5 sec during data runs. These 
included state variables of both airplanes (i.e., a, 3, u, v, w, etc.), 
as well as relative geometric relationships between them. 

The first 2-1/2 days of the experiment were used for familiarizing the 
pilots with the simulator. The schedule during this period was not rigid, but 
an attempt was made to approximately equalize the simulator time for each pilot. 
On the third day, pilot- versus-pilot data runs were begun. Departures and 
crashes were a problem from the beginning. 

The extreme competitiveness of fighter pilots and the lack of life- 
threatening consequences in the simulator caused a large number of runs to ter- 
minate before 3 min had elapsed. About 4-1/2 days were required to complete 
this data set. In fact, in order to ccsnplete the experiment within the con- 
straints of alloted simulator time and available pilot time, 9 runs with dura- 
tions of less than 3 min but more than 2 min were accepted as data runs. 

About 3 days were spent attempting to acquire the 6, 3-min runs for each 
pilot flying against the AML control system. Again, time constraints forced 
the acceptance of several runs of less than 3 min. 

The AML performance model was the last in the series to be run. Only about 
2 days were required to complete this set. Thus enough time was left to fill 
out the control- model data set with 3-min runs. Only the pilot- vers us- pilot 
runs were left compromised by an incomplete set of 3-min runs. Upon completion 
of all runs, pilots were debriefed and their comments taped. 
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PILOTS 


Pilot background was important because the AML maneuvering-performance 
results were to be compared with those of typical U.S. military fighter pilots. 
The background and experience level of the pilots covered a broad range. All 
four were currently flying F-4's, which was a requisite of the experiment. Two 
were U.S. Air Force pilots. The other two were U.S. Navy pilots. Two were 
instructor pilots. Flight experience varied from about 4000 hr in many types 
of aircraft, including helicopters, to only about 300 hr primarily in the F-4. 
The maximum F-4 experience was about 1800 hr. 

Pilots were given a fairly ccmprehensive briefing, including the operation 
of the DMS and AML, the purpose of the study, methods of data analysis, and 
general procedures to be followed throughout the experiment. For instance, 
they were told to request rest periods as needed regardless of the run schedule. 
There were asked to make at least one nondata run each time upon entering the 
simulator to allow their eyes to accommodate to the simulator lighting. They 
were also instructed to tell the computer operator when they were ready to begin 
data runs. 

No particular maneuvers were specified for the pilots to use. They were 
left entirely to their own resources to devise and use the most effective maneu- 
vers. However, they were told that no simulated weapons would be fired and that 
runs would not be terminated when a pilot acquired the necessary conditions to 
fire a particular weapon at the other. Rather, their objective was to acquire 
and retain the best possible positional advantage relative to their opponent. 
Within this context, then, certain entries into weapon zones might be examined; 
however, these would function only to give a measure of the degree of positional 
advantage. 


RESULTS AND DISCUSSION 

Several methods of analysis and evaluation were applied to the data. Gen- 
erally, these were comparisons of computed parameters which quantify various 
types and degrees of positional advantage attained by the pilots or the AML- 
controlled airplane during a run. Some statistical tests on these parameters 
were also made but were of very limited scope and depth. It is important that 
the reader not attach undue significance to these statistical tests, however. 

No absolute performance indexes have been found which measure the outcome of 
one-on-one simulated ACM. The parameters used in this paper should be consid- 
ered merely as indicators of relative performance. Maneuvering in air combat 
is too complex and includes too many variables to be measured well by a single 
index. Thus, a ccmprehensive statistical analysis of a particular indicator 
could possibly produce erroneous results. However, it is believed that by con- 
sidering all of the analyses which will be presented here, a valid assessment 
of the results can be obtained. 

To permit direct comparisons with other data runs, some data were adjusted 
from the nine pilot-versus-pilot runs that lasted less than 3 min. The 
adjusted parameters are simply the accumulated times during a run that the 
pilot was able to remain within certain geometric zones defined by relative 
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headings, range, and the like. They were adjusted by multiplying the observed 
parameters by the inverse of the actual run length to 180 sec. For example, 
assume that a run lasts 120 sec and that pilot X accumulates 30 sec of param- 
eters Y. For comparison purposes, the effective accumulated seconds of param- 
eter Y is 

180 

x 30 = 45 sec 

120 


Before discussing individual performance indexes, time histories of basic 
run data will be presented. 


Time Histories 

From data recorded on magnetic tape, time histories of several airplane 
state variables, as well as computed geometric relationships between the air- 
planes, were plotted. Four of these variables that were found to be most 
descriptive of the ACM engagements are shown plotted for each run in figures 7 
to 78. Of f-boresight angle X is the angle between the X body axis of a 
combatant's airplane and the line of sight to his opponent. If this angle is 
zero, the combatant is pointing the nose of his airplane directly at the center 
of gravity of his opponent's airplane. As X increases, the combatant points 
farther and farther away from his opponent. The ideal situation for a given 
pilot, for example, is to have his X « 0° while that of his opponent is 180°. 
The other three variables plotted are Mach nixnber, altitude, and range. Fig- 
ures 7 to 30 are pilot-versus-pilot runs. Figures 31 to 54 are pilot-versus- 
AML-control-model runs; and figures 55 to 78 are pilot- versus-AML- performance- 
model runs. These figures provide a broader perspective of the engagements than 
do certain signal- variable sooring criteria to be discussed later. They allow 
the viewer to follow each engagement from beginning to end. He can determine 
when each canbatant had an angular advantage as well as other concurrent aspects 
of the fight which can influence the meaningful ness of the advantage. Also, 
certain tactical maneuvers may be observed. For instance, attempts by one pilot 
to "scrape" the other off by diving toward the ground may be detected in the 
altitude plots. Likewise, overshoot situations may be detected by observing 
the Mach plots in conjunction with the range plots. 

Severed, general observations may be made by comparing the data in the vari- 
ous groups (pilot-versus-pilot, pilot- versus-AML-control-model, pilot- versus- 
AML- performance-model) . Pilots appear to sustain higher altitudes and speeds 
against other pilots than they do against the AML models. The lower altitudes 
observed in the pilot- versus-AML runs were often critical for maneuvering. That 
is, many otherwise optional maneuvers were no longer available because of the 
low altitude. The longest endured critically low altitudes are more noticeable 
in the pilot-versus-AML-performance- model group. 

Ccmparisons of the X plots of the two AML models indicate that the con- 
trol model was generally able to achieve a more desirable angular relationship 
against the human pilots. There are also indications that the control model 
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has superior low-altitude maneuvering capability. These data have been pre- 
sented at this point in the discussion of results to provide the raw results 
upon which many of the analyses to follow are derived. 


Time on Offense With Advantage 

The most often used parameter to evaluate the results of engagements in the 
DMS is time on offense with advantage (TOA) . It is simply the accumulated time 
that one airplane is able to keep its opponent in front of its wing line and 
simultaneously to remain behind the opponent's wing line. Although this param- 
eter covers a wide range of conditions from close tracking to near parallel 
flight at all ranges, it still seems to be a very good indicator of relative 
maneuvering performance. Many DMS studies attest to this fact. 

Figure 79 is a bar chart showing the average TOA acquired by each pilot 
when flying against the other three pilots. The individual- run data are pre- 
sented in table I. 

Figure 80 is a bar chart comparing the individual TOA scores of each pilot 
with those of the performance model against him. The mean TOA of all pilots 
is also shown for comparison with the mean TOA of AML. Figure 81 presents the 
same information for the AML control model. 

Kolmogorov-Smirnov tests for normality were applied to the data of fig- 
ures 80 and 81 . From these, it was concluded that the data could be assumed 
normal, permitting the use of t-tests to determine whether there are significant 
differences between the means of certain data groups. Overall pilot scores as 
well as those of individual pilots were compared with corresponding scores of 
the AML models. The results of these comparisons, including the individual TOA 
scores from which they were derived, are summarized in table II. 

The following observations are noted: 

1 . For pi lot- vers us -AML-performance- model runs - 

The mean pilot TOA is significantly less than the mean TOA of the 
AML (dotted lines of fig. 80) . 

The individual mean TOA of pilots C and E are significantly less 
than the corresponding ones of the AML. 

2. For pilot- versus-AML- control-model runs - 

The mean pilot TOA is significantly less than the mean TOA of the 
AML (dotted lines of fig. 81). 

The individual mean TOA of all pilots except pilot E are signifi- 
cantly less than the corresponding ones for the AML. 

3. The mean TOA of the pilots is approximately the same against both AML 
models. The mean TOA of the control model is much greater than that of the 
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performance model and differs significantly from it (student's t) at the 
95-percent confidence level. 

4. In only one comparison did a pilot TOA exceed that of the AML against 
him. Pilot D was able to accomplish this against the performance model, but the 
confidence level of the significance of the score difference is only 60 percent. 

As previously stated, clear-cut win-loss criteria for simulated ACM engage- 
ments simply do not exist. Even so, it was decided that seme such evaluation 
on a run-by-run basis might provide an enlightening alternative view of the 
data. Thus, a set of criteria believed reasonable, although arbitrary, was 
defined in terms of TOA scores to permit this evaluation. To win a run, a com- 
batant had to have acquired at least 20 sec of TOA and at least twice as much 
TOA as his opponent. All other runs were considered draws. Using these cri- 
teria, it was determined that the control model won 17 runs, tied 5 and lost 2; 
and the performance model won 12 runs, tied 8, and lost 4. 


AML SCORE 

Another parameter often used to analyze DMS results is called adaptive- 
maneuvering- logic score (AMLS) . The AMLS is computed simply by answering the 
following questions just as the AML does to make maneuvering decisions. The 

Answers 
Yes No 


1 . Is opponent in front of attacker? 1 0 

2. Is attacker behind opponent? 1 0 

3. Can attacker see opponent? 1 0 

4. Is opponent unable to see attacker? 1 0 

5. Is attacker in certain cone behind opponent? 1 0 

6. Is opponent not within certain cone behind attacker? 1 0 

7. Can attacker fire at opponent? 1 0 

8. Can opponent not fire at attacker? 1 0 

9. Is attacker closing on opponent? 1 0 

10. Is attacker's line-of-sight angle less than 60°? 1 0 

11. Is attacker's line-of-sight angle decreasing? _J_ £ 

MAXIMUM AND MINIMUM SCORES 11 0 
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questions are evaluated every 0.5 sec of each run and averaged for each adver- 
sary. The value of the AMLS parameter then ranges from 0 to 11. Note that 
this set of questions is smaller than the set on page 7 because it represents 
an earlier version of the AML maneuver-selection criteria that was current at 
the time the DMS postanalysis programs were written. The parameter generally 
varies through a much narrower range than its possible extremes. Typically, 
its values are in the range of 4 to 6. Thus, very small changes can result 
from very large changes in maneuvering performance. An important advantage 
that the AMLS enjoys over TOA is that it accounts for many more aspects of the 
maneuver process than does TOA. On the other hand, the user has greater dif- 
ficulty visualizing what is being measured as well as deciding what level of 
incremental change in the index is significant. Experience has shown that dif- 
ferences of 0.5 or greater generally occur in runs having large performance 
differences between opponents. 

Figure 82 presents results for pilots flying against each other. It cor- 
responds to the TOA data of figure 79. Data for individual runs are presented 
in table I. 

Figures 83 and 84 are bar charts presenting individual comparisons of pilot 
AMLS values with those of the performance model and control model, respectively. 
These correspond to figures 80 and 81 , respectively, which show TOA scores. 

As with the TOA data, the AMLS data were tested using Kolmogorov-Smirnov 
tests. The student's t was used to test both overall and individual pilot 
differences between pilot-AMLS means and those of the AML. Table III contains 
these results. 

The following observations are cited: 

1. Both AML models achieved significantly greater overall mean AML scores 
than did the pilots against them. 

2. The mean differences between the AML scores of the pilots and those 

of the opposing AML model were computed for the data of figures 83 and 84. The 
mean difference for the control-model data was found to be greater than that for 
the performance- model data, the difference being significant at the 85-percent 
confidence level using the t-test. 

3. Only pilot E achieved a higher score than AML against him in the 
control-model runs. This difference was found to be significant at only a 
55-percent confidence level, however. 

4. Only pilot D achieved a higher score than the AML against him for the 
performance- model runs. Likewise, this difference was significant only at a 
60-percent confidence level. 


Missile and Gun Zones 

In order to obtain additional insight into the degree of relative maneuver- 
ing advantage achieved by one DMS opponent over another, simulator runs are 
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often analyzed to determine the time a pilot entered a defined weapon zone and 
how long he remained there. The zones used seldom bear more than a token rela- 
tionship to any existing weapon, as is the case for all zone analysis performed 
in this paper. For this study, three zones were examined. A pilot was con- 
sidered to have satisfied the parameters for entry into missile A zone if he 

reduced his of f-boresight angle to less than 40° and maintained his range 

between 0.2096 km and 5.4864 km. Likewise, he entered missile B zone when his 
of f-boresight angle was less than 20° and the range was between 0.609612 km and 
3.6576 Ion. To enter the gun zone, a pilot's of f-boresight angle had to be less 
than 10° while that of his opponent was greater than 120° and the range less 
than 0.91218 km. As can be seen, the difficulty of achieving these zones 
increases greatly from A to B to the gun zone. 

Table IV contains the time of first entry and the duration of retention 
of each of these zones for each data run, as well as averages of these times. 

Figures 85 to 96 present most of the data of table IV in bar-graph form. In 

figures 85 to 88, the common opponent referred to is a control group consist- 
ing of the four human pilots. The control group was reduced to three pilots to 
obtain data for an individual human pilot since he obviously did not fly against 
himself. Thus, the common opponent for AML was four pilots. The common oppo- 
nent for each pilot was the three other pilots. The reader is cautioned, in 
examining this analysis, that acquiring any of these zones may not necessarily 
have been the pilot's objective. Pilots were not instructed to seek these zones 
nor are the AML goals tailored specifically toward them. In fact, maneuvering 
to acquire the zones in the short term may be counterproductive to attaining 
a more secure long-term advantage. It is believed, in general, however, that 
rapid acquisition of these zones is consistent with getting and keeping the 
greatest positional advantage for the most time. 

Some general characteristics and trends may be observed in the zone data 
of table IV. Generalizations will be grouped to assess: (1) how well pilots 

did against each other and against the AML models, (2) how well the AML models 
did against the pilots, and (3) how well the AML models did relative to one 
another . 

Pilots: 

1 • Pilots were able to first enter missile A zone against each other and 
against the control model in about the same amount of time. 

2. Pilots were able to acquire missile B zone against both AML models much 
sooner than they were against each other. 

3. Pilots were able to remain in missile A zone against both of the AML 
models for approximately the same amount of time. Likewise, they were able to 
remain in missile B zone against both models for another duration of time 
approximately equal for both models. But they were able to remain in these 
same zones somewhat longer against other pilots. 

4. Pilots were able to enter the gun tone against each other in 58.33 per- 
cent of the runs. They made entries against the control model in only 12.5 per- 


17 


I 


I 


cent of the runs and against the performance model in only 4.2 percent of 
the runs. 

AML models: 

1 . Both AML models were able to acquire both missile zones much sooner 
than pilots were able to acquire these same zones against each other or against 
either AML model. 

2. Both AML models were able to remain in both missile zones much longer 
than pilots were able to remain in these same zones against each other or 
against either AML model. 

3. The control model made entries into the gun zone in 12.5 percent of 
the runs. 

4. The performance model made gun- zone entries in 8.34 percent of the runs. 
AML models relative to one another: 

1 . The control model enters missile A zone slightly sooner against pilots 
than the performance model does and remains in both missile zones longer. 

2. The performance model enters missile B zone against pilots much sooner 
than the control model does. 

3. Pilots enter both missile zones much sooner against the control model 
than they do against the performance model. 

4. Pilots remain in both missile zones somewhat longer against the perfor-' 
mance model than against the control model. 

5. Bearing in mind that there were very few gun-zone entries by either 
pilots or AML against each other, out of the number of runs made, pilots made 
three times more entries against the control model than they made against the 
performance model, and the control model made three to two more entries against 
pilots than the performance model did. 


Assessment of Results 

This study has addressed two primary questions: 

1. How well does the AML perform against current U.S. fighter pilots? 

2. How do the two types of AML models compare with one another? 

The performance of the AML models against human pilots was impressive, 
indeed. This was especially true of the control model. The same methods 
generally used to analyze the results of DMS studies were applied to the data 
from this study. In each analysis, the result was the same. The AML models 
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clearly exhibited superior maneuvering. Even before detailed analyses of the 
data begem, the result was obvious from observations of the runs as they took 
place. 


The only real difficulty in addressing the question of how well the AML 
models did against the pilots was determining the amount by which the AML models 
won. The usual methods of analyzing DMS data are generally considered more 
applicable to determining trends in results than to determining the absolute 
magnitudes of relative performance. Thus, this report simply concludes that 
the performance of the AML models was far superior to that of the pilots. 

In all fairness to these pilots, however, the AML did not win all of the 
runs. Furthermore, there was a great deal of variation in pilot performance. 

Not only were seme individual runs won by pilots, but there were even cases in 
which an individual pilot either on the average won his runs against a given 
AML model or came very close. For instance, figures 80 and 83 show that pilot D 
defeated the performance model both on the basis of TOA and AMLS . However, fig- 
ures 81 and 84 show that pilot D did very poorly against the control model with 
the same measures. Likewise, pilot E was only slightly inferior to the control 
model on the basis of TOA and was superior on the basis of AMLS. Conversely, 
he was substantially inferior to the performance model on both bases. Zone data 
(table IV) show similar trends for both these cases. 

Before discussing the relative performances of the two AML models, perhaps 
a better understanding of the data analysis in general can be achieved by con- 
sidering the apparent fact that the character of engagements between human 
pilots and AML models is different from that between human pilots only. 

Against the AML models, pilots produced measures of performance that were less 
variable than those they produced against each other. They much less fre- 
quently gained positions of extreme advantage against the AML models than they 
did against each other. But it is also true that the AML models seldom achieved 
the most desirable positions of relative advantage against the pilots. (See gun- 
zone data.) Looking at the question matrix on page 7, it can be seen that the 
AML gives about equal weight to defense and offense. Note that for most ques- 
tions relating to an offensive advantage, there is a corresponding question 
relating to whether the opponent will gain that same offensive advantage. The 
structure of this question matrix, then, causes the AML to tend to choose maneu- 
vers that are conservative. It will risk giving up one aspect of positional 
advantage only with the reasonable assurance of gaining at least as much off- 
setting advantage. Within the spectrum of maneuvers available to the AML, the 
option to run away does not exist. This is an aggressive constraint. Except 
for this constraint, then, the AML behavior seems to have defensive leanings. 

The human pilot is much less constrained. He can do long-range, as well 
as short-range, planning. He can remember weaknesses of various opponents and 
structure tactics from the very beginning of a fight to capitalize upon them. 

He can also use such tactics as hit and run. On the other hand, pilots are 
sometimes at a disadvantage because of an element of inflexibility that results 
from rigid tactical philosophies. The possible disadvantages of a particular 
tactical philosophy may not readily show up as long as both opponents employ 
it and thereby neutralize the effects of its weaknesses. Along with the pilot's 
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larger array of optional maneuvers go more ways to make disastrous mistakes 
and to capitalize on those of others. Thus, often pilots maneuver to the most 
extreme positions of relative advantage against each other. On the other hand, 
against the AML, the human pilot's maneuver choices become more limited because 
of the necessity to counter the AML narrower range of maneuvers. Being a deter- 
ministic tactical philosophy, the AML makes no mistakes although the maneuver 
it chooses may not necessarily be effective. The point is the AML is consis- 
tent, does not give up, continuously implements sound ground-avoidance and anti- 
departure measures, and uses reasonably good one-on-one air-combat maneuvers. 

An opponent possessing those characteristics has a tremendous psychological 
advantage in that it does not afford the opposing pilot many mistakes. 

Even when a pilot is able to achieve a fairly good positional advantage 
over the AML, it is difficult for him to carry through to a good tracking solu- 
tion because the AML models are extremely difficult to track and easy to over- 
shoot. Likewise, perhaps because of their conservative maneuvering, the AML 
models also seem to have difficulty maintaining extremes of positional advan- 
tage against pilots. 

Formulating a fair assessment of the relative maneuvering effectiveness of 
the two AML models against human pilots is difficult. Part of the assessment 
can, of course, be done in a straightforward manner since both models were oper- 
ated against human pilots as a common opponent. Examinations of figures 79 
to 96 and the question set on page 1 5 reveal that, in an overall sense, all 
three primary measures of performance (TQA, AMLS, and zone analyses) indicate 
that the control model outmaneuvered the pilots by a much greater difference 
than the performance model did. What is not known is how much of this result 
was produced by the order in which these models were flown against the pilots. 
However, the fact that the control model was presented to the pilots first and 
also defeated them by a much greater margin has positive implications. Even if 
the learning effects were large, the relative performance adjusted for learning 
effects would still probably show the control model to be at least as good as 
the performance model. As long as the control model performs at least as well 
as the performance model, it seems reasonable to assume that the performance 
model has not been "cheating." Any maneuvers that the performance model may 
have used against pilots which the pilots could not also use would not have made 
the performance model win by any greater amount than the "uncheating" control 
model would have wort by anyway. 

Seme evidence exists that pilot learning contributed to the performance 
difference between the two models. Since both AML models are driven by identi- 
cal maneuver logic, their performance against the same opponent should differ 
only to the extent that the ability of the two models to implement control 
commands differs, it seems reasonable to expect the performance model to 
execute control commands faster and more precisely, especially under low-speed 
conditions, because its attitude-control authority is constant, its modeling 
scheme does not account for the influence of dynamic pressure on control author- 
ity. Only at high speeds should the control model have superior attitude con- 
trol. Of course, one would expect this to produce a result opposite from that 
obtained. 
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The fact that the mean pilot TOA was about the same for both models but 
that the mean AML TOA was much lower for the performance model than for the 
control model suggests that the pilot's defensive tactics improved by the time 
the performance-model runs were made. 

A comparison of AMLS parameters shows that the pilots' mean score improved 
frcm the control-model runs to the performance-model runs while the mean score 
of the AML models worsened. That both scores changed is to be expected and is 
consistent with TOA results since AMLS parameters are composite measures of 
both offense and defense. Although the AMLS parameters indicate that the per- 
formance difference between the models is large, it yields little, if any, 
information about the contribution of learning effects to the difference. 

In an attempt to gain further insight into what caused the AML scores to 
change as they did, the AMLS parameter was slightly modified and recomputed. 

The objective was to divide the total score into offensive and defensive compo- 
nents to see how the constituents changed. To do this, question 9 of the ques- 
tion set on page 1 5 was omitted because it did not seem to reasonably fit into 
either an offensive or defensive classification. The remaining 10 questions 
were divided into 5 defensive and 5 offensive ones. Offensive and defensive AML 
scores were computed and tabulated (table V), along with a recomputed AMLS based 
on the 10 questions. 

Offensive ; 

1 . Can attacker fire at opponent? 

2. Is opponent in front of attacker? 

"S 

3. Is attacker behind opponent? 

4. Is attacker in certain cone behind opponent? 

5. is attacker's line-of- sight angle less than 60°? 

Defensive : 

1 . Can opponent not fire at attacker? 

2. Can attacker see opponent? 

3. Is opponent unable to see attacker? 

4. Is attacker's line-of- sight angle zero or decreasing? 

5. Is opponent not within certain cone behind attacker? 


It appears that the defensive AMLS generally varies about some nominal 
value by only small increments although these variations may be meaningful. 
The offensive AMLS, on the other hand, undergoes much larger variations. 
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These results show that the pilots were able to improve their offensive 
AMLS against the performance model relative to the control model by about 
8.4 percent while causing the AML offensive score to decrease by 10.7 percent. 
The increase in pilot offensive score along with the decrease in the AML offen- 
sive score from the control-model runs to the performance- model runs reinforces 
the idea that the primary difference between outcomes of the runs against the 
two models was caused by the pilots' learning how to neutralize some of the AML 
offense. The increase in the pilot's offensive score may be accounted for by 
the fact that, in sane runs, the pilots were able to defend themselves well 
enough to allow implementation of sane successful offensive maneuvering. 

Although every way that the data have been analyzed shows the control model 
to have done at least somewhat better than the performance model, it is felt 
that the data available fron this study are insufficient to conclude that either 
model possesses inherently superior maneuvering performance with respect to the 
other. 


If, in fact, neither model has any real maneuvering advantage over the 
other, a very good situation exists. Much simpler airplane models requiring 
only performance data may be quickly assembled for use in air-canbat simulation 
studies or pilot training. Control models will, however, continue to benefit 
frcm enhanced pilot confidence and a sense of fair treatment. Care should be 
exercised in assuming that the results of this study apply to any other simu- 
lated airplanes. The relative maneuvering performance of the two different 
models may be very different for airplanes other than the F-4. 


PILOT COMMENTS 

During the course of this study and immediately following its completion, 
several comments were extracted from discussions in which the pilots gave their 
opinions of the AML programs. The pilots' opinions were directed toward the 
overall operation of the real-time programs, the individual characteristics of 
each program, and how the programs compare with each other. 

The pilots were in mutual agreement in their opinions on the overall oper- 
ation of the AML programs. They felt that the AML had tremendous potential as 
a training aide. They felt it could increase the proficiency of the pilots 
since it was always aggressive and appeared to do the right things. However, 
the pilots were of the opinion that the AML did not always utilize the full 
capability of the controlled airplane at lew altitude. In addition, they all 
believed that the AML did not adequately take advantage of opportunities to 
improve its angular advantage in many situations. It is believed these weak- 
nesses can be improved with refined low-altitude tactics and with more accurate 
predictions of the relative state between the controlled airplane and its 
opponent. 

In discussing the individual characteristics of each program, the pilots 
noted that the control-model program was more likely to work the vertical 
whereas the performance-model program stayed low and tended to minimize its 
vertical maneuvering. They felt that the control-model program performed more 
rolling- type maneuvers than the performance model. One pilot thought that the 
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performance model was unable to perform many of its selected roll maneuvers 
because of its low altitude. A complaint was registered against the 
performance- model program. Two of the pilots felt that the performance model 
occasionally made unrealistic changes in attitude. This could possibly be 
caused by the incomplete modeling of the airplane's body rates. The other two 
pilots did not express their opinions on this subject; however, all the pilots 
agreed that the control model was smoother than the performance model in atti- 
tude changes. 

The pilots had varying opinions as to which of the two models was the 
better opponent. One pilot said it was difficult for him to say. Two of the 
pilots, however, made more definite statements on the subject. One said that 
the performance model was better because it was harder to predict and appeared 
to have better control of the fight. The other pilot felt that the control 
model was easier to predict and would, therefore, allow him to enjoy more suc- 
cess in combating it. According to the opinion of this last pilot, one could 
deduce that the overall performances of the two models were about equal. He 
stated that the control model was more difficult to stay behind than the per- 
formance model; however, it was more difficult to get behind the performance 
model. 


FUTURE APPLICATIONS 

It is recommended that several areas of research be pursued with the AML. 

In order to continue to improve the tactical logic, the capability of riding 
in the AML-controlled airplane needs to be implemented. All that is required 
is to drive the display systems in the second DMS sphere. This simple task 
would permit fighter pilots to observe the functioning of the AML from two 
ppints of view, thus supplying them with a much broader insight into the 
strengths and weaknesses of the logic. This information could be supplied to 
engineers who would use it to improve the AML. 

Through advanced displays, the AML could provide suggested maneuvers to 
fighter pilots. The AML could even assume control of a fighter airplane in 
emergencies such as complete loss of pilot consciousness during sustained 
high g maneuvering. 

The AML might be used to guide remotely piloted vehicles that act as wing- 
men for fighter airplanes. Studies need to be done to assess the most effective 
ways of employing vehicles of this type and to determine how the human pilot 
would coordinate and coimiunicate with such partners. 

The principles of the AML need to be extended to the multiaircraft environ- 
ment. Here one might envision a two- tier logical process. Rather than evaluate 
maneuvers against only one airplane, it would evaluate a set of maneuvers for 
each of n hostile airplanes, choosing the most effective to be used against 
each. These n chosen maneuvers would then be reevaluated on an overall level, 
selecting the one which exposes the AML airplane to the least threat from n 
hostile airplanes and which, in addition, provides the greatest opportunity for 
aggressive activity. Furthermore, if more than one AML partner is present, the 
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maneuvers could be assessed on an even higher (third) level, allowing one AML- 
controlled airplane to come to the aid of another. 


CONCLUDING REMARKS 

A digital-computer program known as adaptive maneuvering logic (AML) has 
evolved from a batch-processing program driving greatly simplified airplane 
math models in interactive air-ccmbat maneuvers against each other to a highly 
refined real-time program driving a complex math-modeled P-4 airplane in suc- 
cessful, interactive, simulated air-canbat maneuvers against skilled human 
pilots. If the pilots in this study had encountered, for the first time, real- 
world hostile airplanes controlled by a logic that performs even close to as 
well as the AML, they would have suffered near total defeat. Of course, many 
problems would have to be solved to implement the AML in real-world airplanes. 
However, the technology to solve these problems will soon be available if it 
is not already. Three primary technical problems must be overcome. First, 
sensors must be developed that are capable of determining with spherical cover- 
age the relative position of the opponent in space. Although the present AML 
programs use perfect sensor information, they probably can perform well with 
much less accurate information. Studies need to be made to assess this require- 
ment. Second, the airplane would need computers with sufficient capacity to 
perform the AML functions. Again, the magnitude of this requirement has not 
been determined. The requirements would be not nearly so great, however, as 
those presently to drive the DMS-AML program, which must carry out many more 
functions than just AML computations. Third, the AML now controls its airplane 
in an idealized aerodynamic environment. There are no winds, no gusts, and the 
like, and the airplane model obeys exact equations of motion. The control sys- 
tem would have to be reconfigured to function in a real-world environment. 

Although not proven by this study, the results indicate that performance 
models which are much more readily programmed and which require fewer computa- 
tions than control models may be sufficient for many applications of the AML. 
This study has also greatly enhanced confidence in the results obtained from 
earlier studies that used performance models. 


Langley Research Center 

National Aeronautics and Space Administration 
Hampton, VA 23665 
July 18, 1979 
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APPENDIX A 


PERFORMANCE- MODEL MODIFICATIONS 

Bobby J. Glover 
Kentron International, Inc. 

Appendix A covers modifications made to the real-time and batch AML 
performance-model programs. It should be noted that this appendix does not 
cover all the changes made to the original version of the AML. Instead, it pri- 
marily covers the major changes that were made after the contractor reports of 
references 1 and 2. Many other changes such as plotting capabilities, printout, 
minor logical manipulations, and program sequencing are not discussed. 

The modifications are discussed alphabetically with respect to the sub- 
routine that contains then even though some are not directly applicable to the 
basic function of the subroutine. The discussions may appear brief since no 
attempt was made to cover previously published material. Therefore, it is 
necessary for the reader to have a good understanding of references 1 and 2 
before he can fully comprehend the meaning and impact of some of the changes. 

A current program listing would also be a valuable tool when reviewing the 
changes. The AML computer programs LAR-12301 and LAR-12553 are available from 
COSMIC, 112 Barrow Hall, University of Georgia, Athens, GA 30602. 

Each individual change made to the AML programs was verified in either the 
real-time or batch mode, and generally was verified by both. Evaluation of the 
results revealed that most changes improved the AML capability. 


Subroutine AERF4 

Although subroutine AERF4 retains the same name used in previous versions 
of the AML programs, the interpolation intervals and variable output have been 
reconstructed. The subroutine now performs a linear interpolation and outputs 
the variables given in the table on the following page. 

With the exception of the thrust variables, the variables listed are either 
new or their data arrays have been expanded or modified to allow program consis- 
tency with airplane simulation programs on the Langley differential maneuvering 
simulator (DMS) . The new variables are maximum and sustained lift coefficients, 
corner velocity, and angle of attack. The corner velocity is the Mach nunber 
at which the controlled airplane can achieve its highest turn rate. 

The tables of maximum and sustained lift coefficient were installed in 
AERF4 to replace the tables of maximum and sustained load factor. The storage 
interval, along with the linear interpolation, had caused excessive drag, espe- 
cially in the Mach regime between 0. and 0.5. In this regime, load-factor data 
were stored at Mach numbers of 0.2 and 0.5. Load factor was held constant below 
M * 0.2 and computed by linear interpolation between Mach numbers of 0.2 
and 0.5. It was discovered that by holding the load factor constant at 
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Variable 

FORTRAN 

name 

Function of - 

Military thrust 

TMILX 

h, M 

Afterburner thrust 

TABX 

h, M 

Idle thrust 

TIDLEX 

h, M 

Speed-brake drag 

CDSBX 

M 

Corner velocity 

OORVX 

h 

Dive recovery angle 

RECANX 

h, M 

Maximum lift coefficient 

TCLMX 

h, M 

Sustained lift coefficient 

TCLSX 

h, M 

Coefficient of drag 

CDX 

C L , M 

Angle of attack 

ALFATX 

C L , M 


M = 0.2, incorrect lift coefficients C L were computed and used in the table 
lookup for drag. The C L used in the table lookup was computed by the follow- 
ing equation: 


(W) (Load factor) 



Since q decreased with decreasing Mach number while load factor, W, and 
S remained constant below M = 0.2, C L increased, which caused drag to 
increase (C D increases with Cj,) . An atttempt was made to correct this prob- 
lem by computing load factors between M = 0. and M * 0.2 with the same lin- 
ear interpolation scheme that is used to compute load factors between M = 0.2 
and M = 0.5. However, this linear interpolation still did not provide the 
required accuracy. Ultimately, the load-factor tables were replaced with tables 
of lift coefficients. The value of C L was assumed to remain constant below 
M<* 0.2. This change has proven quite satisfactory. 

The corner velocity, which was incorporated into this subroutine, is tabu- 
lated as a function of altitude and is used by the throttle-control subroutine 
in determining the throttle setting during normal maneuvering conditions. The 
throttle logic sets the throttle position to idle, military, or afterburner 
thrust in order to maintain the designated corner velocity. Consequently, the 
airplane's speed control is aimed at a level near that at which it can achieve 
its highest turn rate. 

The table lookup for a as a function of Cj, and M replaced the a 
computation, which was based on a linear relationship between C L and ot by 


a 


c Jl 
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with C La being a function of Mach and altitude. This computation, due to the 
linear assumption, caused incorrect values for ot at high and consequently 

created a misalinement problem in the body axes of the controlled airplane. 

The tabulated values of a as a function of C L and M not only eliminate the 
linear assumptions but also provide a better definition of a over a broader 
regime. 


Subroutine EQMOTT 

EQMOTT is one of the more important subroutines contained in the perfor- 
mance versions of the AML programs. It has the responsibility of executing the 
selected maneuvers. Force and attitude equations are driven by its controlled 
bank, load factor, and thrust designations until the desired situation is 
achieved. Digital filtering is provided each cycle for the rotational rates 
of the controlled airplane's body axes. This smooths the attitude transition 
of the controlled airplane when it is commanded to change its maneuver plane. 

Several modifications were made to this subroutine to enhance the simulated 
flight characteristics of the controlled airplane. The subroutine is now more 
effective in executing the selected maneuvers, which increases the overall per- 
formance of the maneuver logic. Because of the length of the modifications and 
the involvement of the logic, only the basic concepts of the changes and their 
contributions to the performance or realism will be discussed. 

Computati ons of aerodynamic quantities and roll angle .- These computations 
pertain primarily to the transition mode in which the controlled airplane rolls 
from one maneuver plane to a new commanded plane. Prior to modification, the 
magnitude of lift, angle of attack, drag, and thrust was held constant during 
transitions between maneuver planes. This reduced the validity of the model 
and created unnecessary discrepancies between the Euler angles of the maneuver 
plane and the Euler angles of the controlled airplane body axes. These discrep- 
ancies prevented the maneuver logic from selecting the best maneuver for the 
airplane. In order to correct this problem, the flow of the subroutine was 
rearranged. The computations of the previously mentioned quantities are cur- 
rently performed during every iteration. 

The bank-angle computation, which is also performed during the transition 
mode, currently updates the basic roll- angle command by the amount that the con- 
trolled airplane rolled during the previous program iteration. Previously, 
maneuver- plane transitions were performed by driving the attitude equations 
with a constant roll rate for a computed number of program iterations. The 
number of program iterations required for a particular transition was dependent 
on the magnitude of the desired roll-angle change and on the maximum roll rate 
of the controlled airplane. This technique created maneuver- plane errors when 
the airplane was unable to achieve the commanded roll angle. The present roll 
transition process drives the attitude equations to null the error between the 
commanded roll angle and the airplane's current roll angle. 

C L filter.- Jerky and erratic motions had been noticeable in the real-time 
flight trajectories of the controlled airplane when it was required to perform 
a maneuver with a much higher or much lower than the previous one. It was 
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determined that maneuvers of this nature caused large instantaneous changes in 
the controlled airplane's loading, thus changing its state in an unrealistic 
manner. To correct this problem, logic for filtering the lift coefficient was 
developed and installed in EQMOTT. 

The basic concept of the filter is to increase or decrease the Cl of the 
previous maneuver by small constant increments until the required Cl is 
reached. This smooths the flight trajectory by eliminating the instantaneous 
load changes. 

Situation energy management.- The energy-management function slightly 
digresses from the basic function of EQMOTT. It is part of the tactics logic; 
however, due to the accessability of the required variables, the logic was 
installed in EQMOTT. 

The energy-management logic reduces the magnitude of a commanded load 
factor for certain situations that occur after a near head-on pass. The magni- 
tude of the load-factor reduction is dependent on the type of airplane that is 
modeled. However, in most cases, the level will be between what the airplane 
can sustain and its maximum capability for a given condition. This allows the 
airplane better management of its energy and at the same time provides a posi- 
tional situation which is as good as the situation achieved when the airplane 
utilized its full load-factor capability. 

After a near head-on pass, the commanded load factor is reduced (1) when 
the two airplanes have moderately large separations without the opposing air- 
plane having a small X and (2) when the airplanes' separation is not large, 
but they have large separation rates and large values of X. 

"Over- the- top" problem.- For some time, it was observed that the AML per- 
fnn rian rp 'model occasionally had difficulty completing vertical loops. It would’ 
appear to lose interaction with the opponent and roll in a confused, unordered 
fashion, often resulting in a hammerhead stall. The airplane just could not 
get "over the top." If the plane of the loop was inclined slightly from the 
vertical, however, the problem was not apparent. 

The mechanics of the over-the-top problem are involved and difficult to 
discuss with reasonable brevity. Especially with the performance model, consid- 
erable background familiarity with the way in which the equations of motion are 
modeled is required to fully understand the problem and its solution. For more 
complete information, the reader is referred to pages 19 through 26 of refer- 
ence 2, as well as to an actual current FORTRAN listing of subroutine EQMOTT. 

A very brief background will be presented here, but it is considered mini- 
mal, at best. Recall that the performance model utilizes a number of approxima- 
tions to characterize its equations of motion. It must do this because it has 
no means of implementing commands from the logic by deflecting control sur- 
faces. It simply assumes that the commanded bank angle and load factor can be 
achieved and essentially places the airplane in the commanded attitude. To 
maintain a degree of reality and continuity in the attitude motion, rotations 
about the body axes are driven by commanded body rates that are filtered and 
limited. As has already been discussed, the angle- of- at tack change is smoothed 
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indirectly by filtering changes in lift coefficient. In all computations that 
are made to drive the airplane's attitude, the angle of sideslip is considered 
zero. This does not mean that it actually is zero, only that it is not 
accounted for in determining attitude driving rates. 

Since the velocity vector is very accessible, it provides a good reference 
on which to base many computations. Thus, an axis system called the moving- 
maneuver- plane (MMP) axis system is defined with the velocity vector serving 
as its X-axis. Rotations made about the velocity vector by using the "right- 
hand rule" define maneuver planes in which the airplane is controlled to fly. 
During transitions from one commanded bank angle to another, the airplane is 
assumed to fly in intervening "instantaneous maneuver planes," with its wings 
perpendicular to them. The transition continues until the maneuver plane cor- 
responding to the new commanded bank is reached. Both the X-axis and the Z-axis 
of the MMP system lie in the maneuver plane. In general, the axis system of 
airplane body relates to the maneuver plane in exactly the same way. That is, 
its X- and Z-axes are also contained by the maneuver plane. Since the sideslip 
is assumed to be zero, the velocity vector and the X body axis are always sep- 
arated by a. Lift and drag forces are set up directly in the MMP with thrust 
forces transformed to it by a. The direction cosines of the MMP axis system 
are used to transform the forces to the inertial axis system. 

Concentrating on the rolling process, now follow through a cycle to see 
how the driving body rates are determined. To effect the roll maneuver, the 
airplane is assumed to roll at some fixed rate. From this, a fixed number of 
degrees of roll per iteration are computed. This incremental change in bank 
angle then becomes the goal or desired change during the current program itera- 
tion. Since the incremental bank of the airplane will be the same as that of 
the instantaneous maneuver plane, a unit vector along the X body axis may 
readily be transformed to the MMP system. It will project along the X and 
Z fWP axes by the cos a and sin a, respectively. The value of a is 
determined from a table for the current conditions of Mach number and lift 
coefficient. The direction cosines of the MMP system are used to ultimately 
determine the inertial components of the unit vector. These are then used to 
compute the Euler angles 0l> and 0) of the attitude to which it is desired to 
drive the body during the current program iteration. From these desired Euler 
angles, the known present ones, and the iteration rate, the required Euler rates 
(4>, 0, and <t>) are computed. They are transformed to body rates (p, q, 
and r), filtered, and limited. Finally, the body rates are used to update 
body quaternions which yield the airplane's new actual attitude at the end of 
the current iteration. 

The scheme functions well as long as the Euler roll angles of the MMP and 
the body are approximately the same. It should be noted, however, that although 
an incremental roll about the X-axis of the MMP is exactly the same as that 
required about the X body axis of the airplane in order to maintain its wings 
perpendicular to the maneuver plane, the corresponding Euler roll angles of the 
MMP and the airplane body are not equal in the general case. They may both 
legitimately be zero, or they may both legitimately be ±180°. Otherwise, they 
are equal only if the angle of attack is zero. The performance model, however, 
makes the assumption that they are always equal. The assumption is a good 
approximation throughout most of the flight regime. Although the accuracy of 


29 


i 

!. 



APPENDIX A 


the approximation decreases as a increases, a never exceeds 30°. Thus, by 
itself, it creates no great problems. The accuracy of the approximation also 
decreases as the magnitude of the pitch angle of the velocity vector increases. 
Severe breakdowns of the approximation take place if both the velocity vector 
and the X body axis are contained by a vertical or near vertical plane (per- 
pendicular to the X-Y inertial plane) and the pitch angle of the velocity vec- 
tor summed with the angle of attack exceeds 90°. A similar situation occurs 
in the vicinity of -90°. Consider, for example, a vertical- plane situation in 
which both the velocity vector and the X body axis have positive pitch angles, 
with the pitch angle of the X body axis being the greater. The Euler roll 
angles of both the MMP and the body axis system must be zero. If the airplane 
continues to pitch upward, it will pass through a pitch angle of +90°. When 
it does, its body-axis roll angle should change from 0° to 180° instantaneously, 
but the roll angle of the MMP obviously should remain zero until the velocity 
vector transitions through 90°. Clearly, this is the worst possible conflict 
between what should happen and the approximation of equal Euler roll angles. 

As previously mentioned in the discussion of this subroutine, two differ- 
ent control modes operate within the routine. In the maneuver mode, the objec- 
tive is to sustain planar flight once the commanded maneuver plane has been 
reached. The desired roll angle of the body axis is determined from the bank 
of the MMP, which is the reference to be maintained. If the geometric condi- 
tions are such that the approximation of equal Euler angles breaks down, the 
body-axis roll is driven in a meaningless way. If the same conditions exist 
in the transition mode which drives the body from one commanded maneuver plane 
to another, an incorrect approximation is made to determine the MMP Euler roll. 
Thus, in one case, the bank of the MMP is correct, but the corresponding bank 
of the airplane is not, and vice versa. Both cases result in unrealistic 
dynamic behavior. The forces acting on the airplane are improperly oriented 
and/or the attitude of the airplane is driven in an improper manner. 

The vertical- plane example cited is a maneuver-mode case. When the body 
axis passes through +90°, the quaternions recognize this and correctly switch 
the body roll to ±180°. The approximation of equal Euler angles, however, 
requires that body roll be equal to the MMP roll which is still zero. Thus, 
large body rates are imposed upon the quaternions to accomplish this. 

In an effort to reduce the problem, a "fix" was devised for the most severe 
cases. Specifically, the vertical- plane case was attacked. Two conditions 
identify the vertical- plane problem. The plane is determined to be vertical 
if a unit vector along the cross product of the X body axis and the velocity 
vector does not project onto the inertial Z-axis. Secondly, for the equal 
Euler angle approximation to break down, the body axis and the MMP axis must 
be on opposite sides of the 90° pitch-angle point. This condition exists if 
their respective Euler yaw angles differ by 180°. Inexact conditions are 
applied to the identification of the vertical-plane situation to cause planes 
inclined slightly to either side of the vertical to be identified and treated 
in the same manner as the vertical planes are treated. 

If these vertical- plane conditions are present, the Euler roll angles are 
recognized to be different and are assigned corrected values. In addition, no 
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changes in bank-angle commands are permitted until the airplane has maneuvered 
out of the abnormal condition. 

Although this fix has not completely eliminated the problem, it has greatly 
improved maneuvering effectiveness of the performance model. 

The previous over-the-top problem is no longer a noticeable deficiency in 
the model* s dynamic behavior. On the contrary, it is very competitive with 
smooth, realistic attitude motion. Yet it is believed that further improvement 
can and should be made by those who use the program in the future. 

It should be possible to drop the equal Euler angle approximation entirely 
and replace it with expressions which give the second required Euler angle in 
terms of the known one. The attitude_of the MMP axis system may be expressed 
in terms of three Euler rotations from the inertial axis system. From 

the MMP system, the body-axis system may be expressed as a fourth rotation a 
about the MMP Y-axis. Multiplying the four matrices associated with each rota- 
tion (beginning with inertial axes), a three- by- three direction cosine matrix 
locating the airplane's body axes in terms of the inertial axes may be obtained. 
Individual terms in this matrix may be equated to corresponding terms in the 
standard three-by-three direction cosine matrix which relates one axis system 
to another in terms of three standard Euler rotations. Designating the three 
rotations as simply 0, and <j> and considering them to be Euler rotations 

from the inertial axes to the body axes, the rotation 4> may be obtained by 
dividing the second element of the third column by the third element of the same 
column. The expression obtained is 


4> « tan” 1 


cos 0 sin <)) 


(—sin 0} sin a 


Manipulating this expression yields the corresponding expression for the Euler 
bank of the MMP in terms of the Euler bank of the body. 


<t> = 


cos' 


,-l 


-tan © sin ot tan 2 


y cos 


tan 2 <l> cos 2 a + 1 


In the maneuver mode, the expression for should be used. Likewise, 

in the transition mode, the expression for (J) should be used. Some difficul- 
ties may be encountered with the sign of the radical in the expression for <J> 
and will require additional logic. However, since it is known that for angles 
of attack equal to zero, 4> must equal 4>, it can be shown that the positive 
radical will satisfy this condition for positive values of <}> ^ 90°. For 
180° £ 4> > 90°, the negative radical should be used. If $ is negative, <}) 
will also be negative. Thus, should be limited such that 180° £ 4> > -180°. 

The |<t>| should be substituted into the expression for 4>. Finally, the 
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computed (j) should be given the same sign as 4>. Similar reasoning may be used 
to resolve difficulties encountered in computing 4> fran <j>. Seme problems may 
also still occur with the Euler angles at pitch angles of ±90°, but a fix simi- 
lar to that in the present program should be able to handle it. 


Subroutine EXTRT 

Subroutine EXTRT predicts the opponent's position, velocity, and attitude 
at the end of a preselected time in the future. The technique involves making 
a polynomial curve fit through the present and two past points along the oppon- 
ent's flight path. These points are 1 sec apart and represent the opponent's 
position in three dimensions. Once the polynomials are obtained, the opponent's 
state variables are extrapolated to a designated time. 

It was determined by evaluating real-time and batch- processing data that 
maneuvering of the AML-controlled airplane is enhanced by making both the oppon- 
ent's extrapolation time and the controlled airplane's prediction- time variables 
which are dependent on the relative states of the airplanes. In effect, depend- 
ing on the values of the times, it can be assumed that this technique can be 
used to make the controlled airplane either lead, lag, or purely pursue the 
opponent. Since the lead, lag, and pure-pursuit situations have previously been 
discussed in the body of this report, they will not be discussed as such here. 
Instead, the conditions are discussed for changing the extrapolation time and 
the values obtained. It must be remembered that the controlled airplane's pre- 
diction time is varied in conjunction with some of the opponent's extrapolation 
times. The discussion of the variable prediction-time logic for the controlled 
airplane is contained in subroutine REACTT. 

The values of the opponent's extrapolation time were chosen by evaluating 
the performance of the AML while varying the extrapolation time in conjunction 
with various range and relative deviation-angle conditions. A situational type 
logic controls the various values that can be assigned to the time. The logic 
keys on range and the deviation angles of the opposing airplane in determining 
if the extrapolation time is to be changed from its nominal setting. Deviation 
angle is defined as the angle between the line of sight and the velocity vector 
of the airplane. Depending on the value of the previous variables, the extrapo- 
lation time is allowed to vary between a minimum of 1 sec and a maximum of 
4 sec. The 4-sec extrapolation is used for ranges greater than 1 524 m and for 
situations in which the AML has achieved a fairly good tracking solution. The 
minimum value of 1 sec is selected when the range is below 304.8 m and the 
opponent has an angular advantage. Once the range increases and the opponent's 
angular advantage decreases, the extrapolation time is increased to 2 sec. In 
case none of these situations exist, the extrapolation time retains its nominal 
setting (1.5 sec) . 


Subroutine GETRXN 

GETRXN uses discrete maneuver-plane rotation increments (ROTNCT) to assign 
trial maneuver planes for the controlled airplane. ROTNCT is commonly referred 
to as the angle between maneuver planes. The value of ROTNCT depends on the 
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number of optional maneuver planes (NTILTT) per quadrant. Since the maneuver- 
plane designations start at zero, the angle between planes is computed as 


ROTNCT 


(;)/' 


(NTILTT + 1) 


During evaluations of the real-time control- and performance-model pro- 
grams, NTILTT was varied in an effort to assess its effect on the maneuvering 
of the controlled airplane. The nominal value of NTILTT had always been 8, 
which established 10° between maneuver planes. Performance evaluation showed 
that the maneuvering capabilities of the controlled airplane were degraded when 
the angle between maneuver planes was decreased below the nominal value (NTILTT 
was increased). As the angle was decreased, the trial maneuvers became more 
clustered about the maneuver currently being performed and about the plane 
nearest the opponent. Since this situation was undesirable, the angle between 
maneuver planes was increased in several small increments beyond the nominal 
setting until a more satisfactory value was determined. This occurred when the 
angle between maneuver planes was 12.86° (NTIL1T = 6). Angles greater than 
this limited the airplane's capabilities by reducing the number of optional 
maneuver planes. Smaller angles caused too much clustering. 


Subroutine REACTT 

REACTT is executed at each decision interval (nominally 1 sec) to deter- 
mine the most promising maneuver for the controlled airplane and to define the 
variables required by subroutine EQMOTT for execution of the selected maneuver. 
It utilizes the output of several other subroutines in accomplishing its pur- 
pose. At each decision interval, the opponent's flight path and attitude are 
extrapolated to a specified time in the future. Next, the controlled airplane's 
flight path and attitude are predicted for each of the trial maneuvers. The 
relative geometry between the extrapolated state of the opponent and the com- 
puted future state of the controlled airplane is then evaluated. Each maneuver 
is assigned a value resulting from the evaluation. The maneuver with the high- 
est value is the one selected to be performed next. 

The changes made to REACTT range from minor logic manipulations to more 
complex logical evaluations and computations. For example, the segmentation 
in the subroutine has been completely eliminated. That is, it no longer 
requires four iterations of the program for completing the computations in the 
subroutine. The subroutine is now always completed in one iteration. The 
incorporation of the AML program on a faster computer made this possible and 
thus eliminated the delay in command to the controlled airplane. 

Currently, the decision interval is not constant. The time between deci- 
sions can now be greater than or less than 1 sec, depending on the situation. 
Dive recovery is one situation which affects it. REACTT contains similar dive- 
recovery logic to subroutine TRXNXT. Once the altitude of the controlled air- 
plane is below 1066.8 m and its flight- path angle is greater than the recovery 
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angle, a dive- recovery maneuver is selected and the time counter for the stan- 
dard decision interval is reset. The prevention of crashes between decision 
intervals is assisted by this logic. 

The decision interval time is also affected by the airplane's attitude. 

As previously discussed in subroutine E(J!0TT, there is an over-the-top problem. 
The airplane retains the previous conmanded maneuver as long as this situation 
exists. This scheme prevents the selection of an improper maneuver and has 
proven to be very effective in handling the situation. 

Several other modifications were incorporated into this subroutine. These 
modifications are not only more complex than the ones previously discussed, but 
they are also more pertinent to specific areas of the maneuver selection pro- 
cess. Consequently, they have been categorized and are discussed in the follow- 
ing sections. 

Initial maneu ve r selecti on.- REACTT saves the opponent's two previous posi- 
tions for use in the future extrapolation of his position, velocity, and atti- 
tude. This presents somewhat of a problem at the beginning of the engagement 
since the two positions do not exist. Previous versions of the AML programs 
dealt with this problem by preventing the AML- controlled airplane from selecting 
a new maneuver for the first 3 sec of the engagement. During this period of 
time, the opponent's positions were stored and the controlled airplane continued 
its initial input maneuver, which was generally straight flight. The technique 
sufficed for head-on initial conditions but was extremely unfavorable for the 
controlled airplane in sane other initial conditions. 

Currently, the opponent's previous two positions are computed with the ini- 
tial velocity components. Naturally, these computations ass une that the oppon- 
ent has been flying along a straight line with constant velocity. However, they 
permit a maneuver to be selected during the first iteration of the program, 
thereby enhancing the maneuver logic for all initial conditions. 

Undesirable extrapolation detection .- The extrapolation-detection logic 
identifies relative situations that cause the extrapolation subroutine (EXTRT) 
to yield undesirable extrapolated variables for the opponent. The problem 
occurs when the opponent is in the rear hemisphere of the controlled airplane 
and the controlled airplane in the front hemisphere of the opponent. In this 
relative situation, it is possible for the AML to extrapolate the opponent's 
position to a point in front of the AML-controlled airplane. When this occurs, 
it is not very difficult for the opponent to stay behind since the trial- 
maneuver selection assumes the opponent is in front. The extrapolation is not 
in error. The relative velocity is such that the extrapolated range is greater 
than the range between the two airplanes. 

To correct this problem, the actual range between the airplanes is com- 
pared with an extrapolated range. If the extrapolated range is greater than the 
actual range, the opponent's position, velocity, and attitude are not extrap- 
olated. The trial maneuvers are evaluated with respect to the present position, 
velocity, and attitude of the opponent. 
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Variable prediction .- In conj unction with varying the opponent's extrap- 
olation time as discussed in subroutine EXTRT, the capability of varying the 
prediction time for the controlled airplane was installed in REACTT. Two situ- 
ations exist for changing the prediction time from its nominal value of 2 sec. 
The time is changed to 6 sec when the range between the airplanes is greater 
than 1524 m and an undesirable extrapolation has not been detected. It is also 
changed to 6 sec for ranges down to a minimum of 304.8 m if the controlled air- 
plane's deviation angle is less than 30° and the opponent's deviation angle is 
greater than 135°. Increasing the prediction time serves a purpose similar to 
that of increasing the extrapolation time of the opponent. 

L oad- factor reducer .- With the exception of the soft-turn and low-energy 
recovery maneuvers, the AML trial maneuvers command the maximum load-factor 
capability of the controlled airplane. Consequently, for hard-turning fights, 
the controlled airplane will sometimes have to resort to low-energy recovery 
maneuvers, forcing it to relinquish any angular advantage it might have 
achieved. To help prevent this problem, the selected trial maneuver is now 
reevaluated to determine if the commanded load factor can be reduced without 
affecting the value of the maneuver. 

The selected triad, maneuver is reevaluated when the range between the two 
airplanes is greater than 1 524 m, the controlled airplane is behind the opponent 
and the opponent is in front of it, straight flight has not been selected, and 
the controlled air pi am e is not in a dive recovery. When these conditions exist, 
two new trial maneuvers are set up, differing from the selected maneuver only 
in load-factor level. Their load-factor levels are 85 and 80 percent of the 
commanded load factor, respectively. The new maneuvers cure then evaluated and 
scored by the same process used for all maneuvers. The two are compared with 
the selected maneuver, and the maneuver with the highest value is selected. If 
two maneuvers have equal value, the one requiring the lesser load factor is 
selected. 


Subroutine STATET 

Subroutine STATET evaluates the relative predicted and extrapolated state 
variables of the AML- controlled airplane and its opponent and assigns a numeri- 
cal score for each trial maneuver. The score or value of a particular maneuver 
is obtained by answering a set of 1 4 questions covering specific areas and quan- 
tities of interest. These include angular relationships, distances, and veloci- 
ties. The questions and a hypothetical score are shown in the question set on 
page 7, with the AML-controlled airplane referred to as the attacker. The ques- 
tions are worded in such a way that they can be answered by either "yes" or 
"no." The yes answers are assigned a numerical value of 1 and the no answers 
are assigned 0. Once the evaluation is canpLete, the answers are summed for a 
numerical score. The score is returned to the calling subroutine and stored for 
comparison with the other tried maneuvers. 

The question set on page 7 depicts the questions which are currently used 
by STATET. They differ fran the original questions in that one question has 
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been deleted and three questions have been added. The deleted question evalu- 
ated the specific energy rate of the controlled airplane. That is, once the 
controlled airplane's specific energy rate was less than -30.48 m/sec, a value 
of 0 was assigned to the question. This at times caused straight flight to be 
selected for the controlled airplane when other optional maneuvers were better 
for the current situation. The question was removed after real-time evaluation 
revealed that all the AML-controlled airplanes were able to perform as well and 
generally better without it. 

Three evaluation- type questions were added to STATET in an effort to 
enhance the maneuver selection process. Two of the questions canbine existing 
questions in the question array to form new questions. For example, looking 
at the question set on page 7, question 9 was worded so that it will be assigned 
a value of 1 when questions 1 and 2 are each assigned a value of 1. Likewise, 
question 12 will be assigned a value of 1 when questions 3 and 4 are each 
assigned a value of 1. This places more emphasis on situations in which the 
AML- controlled airplane has a decided advantage over its opponent. The new 
questions are effective when several of the maneuvers have near equal values, 
with one or more having values of 1 on questions 1 and 2 and/or 3 and 4. When 
this occurs, the new questions will insure that one of the maneuvers with values 
of 1 on questions 1 and 2 anchor 3 and 4 will be selected. 

Question 11 was added to the question array to assist in preventing 
crashes. It keys on predicted altitude of the AML- controlled airplane. For 
maneuvers that have a predicted altitude of less than 91 .44 m, a value of -13 
is assigned to the question. Therefore, the highest value these maneuvers can 
have is 0. When this question is answered yes, a value of 0 is assigned to it 
so it will not affect the maneuver selection process. 


Subroutine THROTLT 

Subroutine THROTLT is called each iteration of the AML programs to deter- 
mine the proper throttle setting for the AML-controlled airplane. The throttle 
setting is designated by the variable TPOST. TPOST is set equal to 0, 1, and 2 
for idle, military, and afterburner thrusts, respectively. Although all the 
trial maneuvers designate afterburner thrust, this subroutine can change the 
throttle setting if the AML-controlled airplane's variables and other geometric 
conditions warrant it. 

The logic which defines the situations for changing the throttle setting 
is completely different fran previous versions of the AML programs. An impor- 
tant feature of the present version of the subroutine is that it now uses 
corner velocity for determining the throttle setting in sane situations. The 
corner velocity, as previously defined in subroutine AERF4, is the Mach number 
at which the controlled airplane can achieve its maximum turn rate. It is com- 
pared with the airplane's Mach number to determine the throttle setting when 
the range is less than 1 828.8 m and neither the AML- controlled airplane nor the 
opponent has achieved a good tracking solution. The logic is designed to set 
the throttle to idle, military, or afterburner, depending on the speed of the 
controlled airplane. If the airplane exceeds its corner velocity by 20 percent. 
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the throttle is set to idle. Likewise, if the airplane is 10 percent below its 
corner velocity, the throttle is set to afterburner position. Military power 
is used when the airplane's speed is between the specified levels. 

Another section of the throttle logic is designed to prevent the AML- 
controlled airplane from overshooting the opponent once it achieves a good 
angular relationship. In this section, a delta range is calculated by using 
the controlled airplane's total acceleration, range rate, and distance from 
the opponent. The delta range serves as a prediction of an overshoot situa- 
tion. If the situation exists, the throttle is set to idle. 

The last section of the throttle logic deals with the reverse of the pre- 
vious situation. It was designed to cause the opponent to overshoot once he 
achieves a good angular relationship and is close behind the controlled airplane 
with a high closure rate. Once again, the idle setting is used when this condi- 
tion exists. 


Subroutine TRXNXT 

Subroutine TRXNXT sets up trial maneuvers for evaluation at each decision 
interval in the AML programs. The number of maneuvers can vary between 1 
and 10, depending on the state of the AML- controlled airplane and its relative 
situation with the opponent. The state of the controlled airplane determines 
the types of maneuvers to be set up. The maneuvers are divided into the cate- 
gories of dive recovery, low- speed recovery or energy conservation, and normal 
conditions. Dive recovery and energy conservation have priority over the normal 
condition maneuvers. They set up trial maneuvers that are designed to either 
prevent the airplane from crashing or from losing so much energy that it can 
no longer maneuver effectively. If neither of these conditions exist, the situ- 
ation is considered to be normal and trial maneuvers are set up based primarily 
on maneuver- plane rotations that are defined by the controlled airplane's 
velocity vector and the extrapolated opponent's position. 

The current version of this subroutine contains several modifications. 

The order of priorities for the categories remains unchanged. However, each 
category contains either improved logical constraints or additional trial 
maneuvers. The dive-recovery section, first in priority, currently contains 
fewer maneuvering constraints for the prevention of crashes. The dive angle 
is not monitored until the controlled airpdane flies below 1066.8 m. Below this 
level, the airplane is commanded to pull up in a vertical plane once its dive 
angle reaches the maximum angle for recovery. These tests sure performed at 
each decision interval in the program. The same tests are performed in sub- 
routine KEACTT to detect crash situations between decision intervals. If one 
is detected, TRXNXT is called and a dive-recovery maneuver is set up. 

The energy-conservation section is another area in which the logical con- 
straints have been improved. The modifications may seem minor upon initial 
investigation; however, real-time performance evaluation of the controlled 
models revealed that they are very important to the overall operation of the 
maneuver logic. 
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The energy-conservation section has the second highest priority to dive 
recovery. Its trial maneuvers are dependent on maximum available load factor 
and the pitch angle of the velocity vector, the load factor being the more 
important variable. Once the maximum available load factor, which is computed 
from the maximum C L , decreases to specified levels, trial maneuvers are set 
up which are designed to assist the controlled airplane in regaining energy. 

The load-factor levels were initially assigned values of 1.5 and 1.0, respec- 
tively. Real-time evaluation revealed that these designated levels placed too 
much restriction on the maneuvering capabilities of the controlled airplane. 
Currently, the load-factor levels are assigned values of 0.5 and 0.25, respec- 
tively. These levels permit a greater degree of maneuvering freedom and at the 
same time allow the airplane to recover from stall situations. 

The energy maneuvers controlled by the velocity- vector pitch angle were 
redefined in conjunction with the newly specified load-factor levels. Originally 
the controlled airplane was commanded to continue pulling maximun load in the 
maneuver plane nearest the opponent if its load factor had fallen below the 
specified level and the flight-path angle was greater than 80°. It was dis- 
covered that the airplane could not always successfully perform this maneuver. 

In many instances, its energy level became so critical that all positional 
advantages, if any, were relinquished. This logic has been eliminated. 

The final category in which modifications were made to TRXNXT is the 
normal- condition section. Several trial maneuvers were added to the existing 
maneuvers in this section. These maneuvers not only increase the maneuvering 
freedom of the controlled airplane but also refine the maneuver selection pro- 
cess. An example of the refinement is depicted by the utilization of the soft- 
turn maneuver. This maneuver, as in previous versions of TRYNXT, is a trial 
maneuver which will approximately result in a trajectory intercepting the oppon- 
ent at its extrapolated position. The soft-turn maneuver is currently the only 
maneuver set up for the controlled airplane when its deviation angle is less 
than 40° and the opponent's deviation angle is greater than 120°. This con- 
straint forces the controlled airplane to perform what is considered the best 
of the trial maneuvers once it has achieved a fairly good tracking solution. 

Four other tried maneuvers were added to TRXNXT for evaluation during nor- 
mal conditions. Three of the trial maneuvers have their maneuver planes defined 
in 90° intervals with respect to each other, with the maneuver pi cine nearest 
the opponent being the plane of reference. That is, their respective maneuver 
plane rotations are 90°, 180°, and 270° away from the plane nearest the oppon- 
ent. These maneuvers prove beneficial once the tactical position of the AML- 
controlled airplane deteriorates to a level at which all the trial maneuvers 
have a low value assigned to them. The controlled airplane is often in this 
situation after a dive recovery or low-energy recovery. Its state variables 
are such that it cannot achieve much success by performing the other trial 
maneuvers. These three maneuvers provide what could be considered alternatives 
in this situation. 

The other trial maneuver added to the normal- condition section is a dive- 
reoovery maneuver. This maneuver instructs the controlled airplane to pull up 
in a vertical plane with maximum load factor. The selection of the maneuver 
from this section could prevent a critical situation in which the airplane 
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would need more time for recovery. It generally will not be selected from the 
normal- condition section; however, it is felt that the airplane should have the 
option of performing this maneuver since crashes have always been somewhat of 
a problem in the AML programs. 
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REAL-TIME IMPLEMENTATION AND REFINEMENT OF AML CONTROL MODEL 

A control system which converts AML commands of bank angle and load factor 
to appropriate aileron, spoiler, stabilator, and rudder conmands was developed 
for the F-4 airplane (as simulated on the Langley differential manuevering simu- 
lator (DMS) ) by Decision Science, Inc. (DSI) under contract to Langley Research 
Center and was delivered in the form of a batch- processing computer program. 

A more detailed description of this program and its design philosophy may be 
found in reference 4. 

At Langley, a number of program evaluations and refinements have been made. 
The system's ability to execute many types of very demanding command sequences 
was studied, along with conditions under which it might be unable to prevent the 
controlled airplane from departing. As expected, large changes in bank angle at 
high pitch angles were often either performed at very low roll rates or the air- 
plane departed. To produce more desirable responses, it was decided to use a 
quasi-sequential scheme of roll and then pitch to effect these maneuvers. This 
problem and the procedures for handling it will be discusssed in more detail 
later. 


When the Control Data CYBER series 175 computer became available for use 
with real-time simulations, the control model was interfaced with the basic DMS 
program. Prior to this time, computing capacity had been insufficient to do it. 

The control model was designed to replace, in a modular form and, as nearly 
as possible, on a one-for-one basis, the performance model already operating in 
real time. This was done to have the program ready as soon as the new computer 
became available and to minimize real-time checkout. Because of this, however, 
the AML-DMS -program is not as efficient as it could be. Much tabulated aerody- 
namic data, as well as routines for solving equations of motion, could be shared 
with the piloted airplane. 

Concurrent with its implementation in real time, the maneuver logic and 
other AML functions independent of the control system itself were updated to 
the level of the latest performance-model program. Most of these have already 
been covered in appendix A and will not be discussed again. 

In its first real-time flights, the control model did fly although it 
often crashed or departed and was of almost no can petition to the human pilot. 

It was only through a long, drawn-out, mostly trial- and- error process in real 
time that the program was refined to be the super competitor that it is. 
Observing the real-time operation of the system was the only practical means to 
gain the needed insight to refine it. Otherwise, major deficiencies remained 
obscure and difficult to detect although they were often corrected by relatively 
simple modifications. In this perspective, then, the system designed by DSI was 
a good one which remains mostly intact. The company did not have access to 
facilities which could thoroughly exercise the program against a human pilot and 
in fact, at the time, neither did Langley Research Center. 
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Sane improvements which pertain strictly to the maneuver logic and not to 
the functioning of the control system itself were also devised during this 
time. These were also included in the performance model and are discussed in 
appendix A. 

The same problem which had for so long plagued the performance-model pro- 
gram was present with the control model, though in a somewhat different form. 

The over-the-top problem was discussed in appendix A. With the control model, 
much of the problem was eliminated by simply referencing the maneuver- plane 
axis system to the X body axis of the airplane rather than to the velocity 
vector. Having a full complement of moment equations, the control model, 
unlike the performance model, does not depend on velocity- vector- based mathe- 
matical approximations to drive its attitude or to determine what its attitude 
is. Thus, it requires maneuver- plane concepts only to choose new maneuvers. 

A body-axis- based maneuver- plane system has worked well for this purpose. The 
primary program changes to make the maneuver- plane — axis- system conversion were 
made in subroutines REACTT and TRYNXT . In TRYNXT, the rotation about the maneu- 
ver plane X-axis to the plane containing the opponent (ROTT) is computed by 
using body angles rather than velocity- vector angles. Also, in the calls to 
NORPLN to get maneuver- plane normals, body Euler angles are now used. The sub- 
routine NORPLN has been itself simplified to use these Euler angles directly 
to compute the direction cosines of the maneuver plane Y-axis. These are equiv- 
alent to the required inertial canponents of the maneuver- plane unit normal. 

Although the maneuver-plane— axis-system change allowed the quaternions 
to do their job without problems of Euler angle incompatibility, a minor prob- 
lem still remained, when the X body axis (now also the maneuver plane X-axis) 
transitioned through a pitch angle of 90°, the reference with respect to which 
the commanded bank was determined changed. The problem was solved simply by 
evaluating and choosing a new maneuver whenever this occurred. In subroutine 
GETGOM, the pitch-angle transition is detected by a change of 180° in body yaw 
between two consecutive program iterations, and a flag is set. This flag sig- 
nals REACTT to initiate an immediate new maneuver selection by using the new 
reference. 

As previously mentioned, under certain conditions the airplane had very 
poor roll response, as well as a tendency to depart. In general, these condi- 
tions were characterized by a lew airplane energy state, high airplane pitch 
angle, high angle of attack, and a large comnanded change in roll angle. The 
general approach to solving the problem was to trade normal acceleration for 
increased roll response by using a modified roll- then- pitch sequence to effect 
commands . Even in the absence of the previously mentioned problems, it may be 
reasonable to attempt to have roll changes lead ;(i. e., be canpleted slightly 
before) pitch changes. This is especially true where the desired change in 
roll angle is large and the desired change in pitch angle is positive. If it 
is assuned that there is only one direction, in terms of a rotation about the 
airplane's body axis, in which a lift vector of given magnitude should be 
directed to execute each elemental maneuver, then directing this vector fully 
along the interim path of bank is probably undesirable. Energy may be wasted 
and the airplane's path of flight temporarily curved in a direction other than 
that upon which the maneuver's choice was based. This is particularly true of 
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the AML because of the way it evaluates and selects maneuvers. No consideration 
is given to transition banking. For evaluation purposes, it is assumed that the 
airplane's flight path will be planar over the duration of the evaluated maneu- 
ver, beginning at the specified bank angle. Thus, lift used before achieving 
the selected bank angle reduces the validity of the maneuver evaluation logic. 

Primarily the modified process of carrying out AML commands was implemented 
in subroutines AtJGCONT, GET COM, and LOCNTR. Because AUGOONT is an entirely new 
subroutine, it is included in a more detailed separate section which follows the 
present one. In GETCOM, provisions were made to reduce the commanded angle of 
attack during large commanded roll changes without regard for the remaining air- 
plane state variables. No action is taken if the commanded roll change is less 
than 30°. If it is greater, the angle of attack (consequently the load factor) 
is reduced in proportion to the amount that the commanded roll change exceeds 
30°. The computations which reduced angle of attack for "violent roll maneu- 
vers" in subroutine LOCNTR have been removed. The change in GETCOM is less 
restrictive but covers the problems previously handled by this logic. 


Subroutine AUGOONT 

Except for the subroutine AMLVS3 which interfaces the AML control-system 
model with the basic real-time MIS computer program, the subroutine AUGCONT is 
the only entirely new subroutine added to the control-system program. The pur- 
pose of the routine is to insure that the commanded bank changes will be per- 
formed at some minimum roll rate. For a given Mach and altitude condition, the 
effectiveness of the ailerons and spoilers can generally be increased by lower- 
ing the angle of attack. Under certain circumstances, the rudder may also be 
used to increase roll response. However, care must be taken in the use of the 
rudder to prevent departures. 

Since both load factor and bank angle are essential to air-combat maneuver- 
ing, the wisdom of trading normal acceleration for improved roll response may 
be questioned by some. The philosophy can which this routine is based is that 
unless the load factor desired is oriented in the proper direction, or very 
nearly so, it accomplishes very little and may actually have a negative tacti- 
cal effect. 

The trade-off is not always made, but only in circumstances in which roll 
authority has decreased to a predetermined minimum. 

If the desired change in roll is less than 5°, the routine does nothing. 

If the desired change is greater than 5°, an estimate of the time t e 
required to complete the change (assuming a minimum desired roll rate of 
20°/sec) is calculated by 

<t> c - 4>p 

t e = 

20 

where 4> c * s commanded bank angle and 4>p is the present bank angle. 
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Then assuming the ailerons and spoilers to be deflected to their maximums 
in a direction to reduce the roll error, the maximum rolling acceleration that 
they can generate for the present Mach number, altitude, and angle of attack is 
canputed from the following equation: 


• * 
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T xx 
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The estimated roll-angle change A<j> E that would result from using this 
rolling acceleration over the time t e is given by 


*<t e )2 

Me “ — - — + p(t e ) 


The estimated roll change is compared with the actual desired roll change 
(<|> c - <J)p) . if A()>£ is greater, no action is taken. Otherwise, the commanded 
angle of attack is reduced by 5° and A<t> E is recomputed and again compared 
with the desired roll change. The process is continued until sufficient roll- 
ing acceleration is obtained to produce the desired roll change in time t e 
or until the commanded angle of attack has been reduced to zero. 

If reducing the angle of attack to zero still does not provide enough roll 
authority, consideration is given to using the rudder to effect the roll. If 
both the angle of attack and angle of sideslip are each below 15°, the rudder 
is deflected to its maximum in the direction to produce the desired roll. This 
is accomplished by setting a flag which signals the subroutine LACTNR to actu- 
ally set the rudder deflection. 

This routine is executed during every program iteration (1/32 sec). The 
original angle-of- attack command is restored before entering the routine on 
successive iterations. Thus, the amount, if any, of angle-of-attack reduction 
is continuously updated. Likewise, the use of rudder is reevaluated during 
each iteration. 
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Figure 2.- Block diagram of two-d|>here Langley differential maneuvering simulator* 
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Figure 3.- DMS-AML performance model 
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ADAPTIVE MANEUVERING LOGIC 



Figure 6.- Adaptive maneuvering logic. 
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Figure 13.- Pilot-versus-pilot data set for run 7 
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Figure 17.- Pilot-versus-pilot data set for run 11 
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Figure 20.- Pilot-versus-pilot data set for run 14. 
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Figure 26.- Pilot-versus pilot data set for run 20 
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Figure 34.- Pilot-ver 
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Figure 36.- Pilot-versus-AML-control- model data set for run 6 
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Figure 46 .- Pilot- versus-AML-control-model data set for run 16 . 
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Figure 49.- Pilot- ver s us -AML-con trol- model data set for run 19. 




D 




































MACH ALTITUDE. KM RANGE. KM DEG 




_ PILOT C 
□ RfIL 






57.- Pilot- versus-AML-perf 
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Figure 59.- Pilot- versus:- AML- performance- model data set for run 5. 
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Figure 63.- Pilot-versus-AML-performance-model data set for run 9. 










Figure 65.- Pilot- versus-AML-performance-model data set for run 11. 
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Figure 77.- Pilot- versus-AML-performance- model data set for run 23 
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Figure 79 •- Comparisons of tine on offense with advantage for 

pilots versus pilots. 
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Figure 80. - Time on offense with advantage for 
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Figure 86.- Average time in zone A for individual pilots and 
AML against common opponent. 
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Figure 87.- Average time of first entry into zone B for individual 
pilots and AML against common opponent* 
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Figure 89.- Average time of first entry into zone A for individual 
pilots versus. AML control model. 
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Figure 90.- Average time in zone A for individual pilots versus 

AML control model. 
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Figure 91 Average time of first entry into zone B for individual 
pilots versus. AML control model. 
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Figure 92.- Average time in zone B for individual pilots versus 

AML control model. 
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Figure 94.- Average time in zone A for individual pilots versus 
i , AML performance model. 
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Figure 95.- Average time of first entry into zone B for individual 
pilots versus. AML performance model. 
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